Dynamically limiting energy consumed by cooling apparatus

ABSTRACT

Cooling methods are provided which include providing: one or more coolant-cooled structures associated with an electronics rack, a coolant loop coupled in fluid communication with one or more passages of the coolant-cooled structure(s), one or more heat exchange units coupled to facilitate heat transfer from coolant within the coolant loop, and N controllable components associated with the coolant loop or the heat exchange unit(s), wherein N≧1. The N controllable components facilitate circulation of coolant through the coolant loop or transfer of heat from the coolant via the heat exchange unit(s). A controller is also provided to dynamically adjust operation of the N controllable components, based on Z input parameters and one or more specified constraints, and provide a specified cooling to the coolant-cooled structure(s), while limiting energy consumed by the N controllable components, wherein Z≧1.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. Ser. No. 13/305,967, entitled“Dynamically Limiting Energy Consumed by Cooling Apparatus,” filed Nov.29, 2011, and which is hereby incorporated herein by reference in itsentirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with Government support under Contract No.DE-EE0002894, awarded by the Department of Energy. Accordingly, the U.S.Government has certain rights in the invention.

BACKGROUND

As is known, operating electronic devices produce heat. This heat shouldbe removed from the devices in order to maintain device junctiontemperatures within desirable limits, with failure to remove heateffectively resulting in increased device temperatures, potentiallyleading to thermal runaway conditions. Several trends in the electronicsindustry have combined to increase the importance of thermal management,including heat removal for electronic devices, including technologieswhere thermal management has traditionally been less of a concern, suchas CMOS. In particular, the need for faster and more densely packedcircuits has had a direct impact on the importance of thermalmanagement. First, power dissipation, and therefore heat production,increases as device operating frequencies increase. Second, increasedoperating frequencies may be possible at lower device junctiontemperatures. Further, as more and more devices are packed onto a singlechip, heat flux (Watts/cm²) increases, resulting in the need to removemore power from a given size chip or module. These trends have combinedto create applications where it is no longer desirable to remove heatfrom modern devices solely by traditional air cooling methods, such asby using air cooled heat sinks with heat pipes or vapor chambers. Suchair cooling techniques are inherently limited in their ability toextract heat from an electronic device with high power density.

BRIEF SUMMARY

In one aspect, the shortcomings of the prior art are overcome andadditional advantages are provided through the provision of a method offacilitating dissipation of heat from an electronics rack. The methodincludes: associating at least one coolant-cooled structure with theelectronics rack for facilitating dissipation of heat from theelectronics rack, the at least one coolant-cooled structure comprisingat least one coolant-carrying passage; providing a coolant loop coupledin fluid communication with the at least one coolant-carrying passage ofthe at least one coolant-cooled structure; associating at least one heatexchange unit with the coolant loop to facilitate heat transfer fromcoolant within the coolant loop; providing N controllable componentsassociated with at least one of the coolant loop or the at least oneheat exchange unit, the N controllable components facilitating at leastone of circulating of coolant through the coolant loop or transfer ofheat from the coolant via the at least one heat exchange unit, whereinN≧1; and providing a controller coupled to the N controllablecomponents, the controller dynamically adjusting operation of the Ncontrollable components, based on Z input parameters and one or morespecified constraints, to provide a specified cooling to the at leastone coolant-cooled structure, while limiting energy consumed by the Ncontrollable components, wherein Z≧1.

In another aspect, a cooling method is provided which includes:obtaining an electronics rack with an associated cooling apparatuscomprising at least one coolant-cooled structure associated with theelectronics rack for facilitating dissipation of heat from theelectronics rack, the at least one coolant-cooled structure comprisingat least one coolant-carrying passage, wherein the cooling apparatusfurther comprises a coolant loop coupled in fluid communication with theat least one coolant-carrying passage of the at least one coolant-cooledstructure, at least one heat exchange unit coupled to facilitate heattransfer from the coolant within the coolant loop, and N controllablecomponents associated with at least one of the coolant loop or the atleast one heat exchange unit, the N controllable components facilitatingat least one of circulating of coolant through the coolant loop ortransfer of heat from the coolant via the at least one heat exchangeunit, wherein N≧1; and dynamically adjusting operation of the Ncontrollable components, based on Z input parameters and one or morespecified constraints, to provide a specified cooling to the at leastone coolant-cooled structure, while limiting energy consumed by the Ncontrollable components, wherein Z≧1.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

One or more aspects of the present invention are particularly pointedout and distinctly claimed as examples in the claims at the conclusionof the specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 depicts one embodiment of a conventional raised floor layout ofan air-cooled data center;

FIG. 2 is a cross-sectional plan view of one embodiment of anelectronics rack with an attached air-to-liquid heat exchanger enhancingcooling of air passing through the electronics rack;

FIG. 3 depicts one embodiment of a data center with a coolantdistribution unit facilitating liquid-cooling of one or moreliquid-cooled electronics racks of the data center, in accordance withan aspect of the present invention;

FIG. 4 depicts an alternate embodiment of a cooling apparatus andliquid-cooled electronics rack, in accordance with one or more aspectsof the present invention;

FIG. 5A is a more detailed, elevational view of one embodiment of theliquid-cooled electronics rack of FIG. 4, and illustrating rack-levelcoolant distribution structures, in accordance with one or more aspectsof the present invention;

FIG. 5B is a partial depiction of a more detailed embodiment of therack-level coolant distribution structures illustrated in FIG. 5A, inaccordance with one or more aspects of the present invention;

FIG. 6 is a plan view of one embodiment of an electronic system layoutfor a liquid-cooled electronics rack, and illustrating multipleliquid-cooled cold plates and multiple liquid-cooled cold rails coupledin fluid communication, in accordance with one or more aspects of thepresent invention;

FIG. 7A is a schematic of a cooled electronic system comprising oneembodiment of a cooling apparatus, in accordance with one or moreaspects of the present invention;

FIG. 7B is a schematic of a cooled electronic system comprising anotherembodiment of a cooling apparatus, in accordance with one or moreaspects of the present invention;

FIG. 7C is a schematic of a cooled electronic system comprising afurther embodiment of a cooling apparatus, in accordance with one ormore aspects of the present invention;

FIG. 7D is a schematic of a cooled electronic system comprising anotherembodiment of a a cooling apparatus, in accordance with one or moreaspects of the present invention;

FIG. 8A is a flowchart of one embodiment of a control process for acooling apparatus, such as one of the cooling apparatuses depicted inFIGS. 7A-7D, in accordance with one or more aspects of the presentinvention;

FIG. 8B depicts one embodiment of a prespecified data structurereferenced by the control process of FIG. 8A, in accordance with one ormore aspects of the present invention;

FIG. 8C is a flowchart of one embodiment of a process for selecting acritical constraint for use in the control process of FIGS. 8A & 8B, inaccordance with one or more aspects of the present invention;

FIGS. 9A & 9B depict an alternate embodiment of a control process for acooling apparatus, such as one of the cooling apparatuses depicted inFIGS. 7A-7D, in accordance with one or more aspects of the presentinvention;

FIGS. 10A & 10B depict an another embodiment of a control process for acooling apparatus, such as one of the cooling apparatuses depicted inFIGS. 7A-7D, in accordance with one or more aspects of the presentinvention;

FIGS. 11A-11D depict a further embodiment of a control process for acooling apparatus, such as one of the cooling apparatuses depicted inFIGS. 7A-7D, in accordance with one or more aspects of the presentinvention; and

FIG. 12 depicts one embodiment of a computer program product or acomputer-readable storage medium incorporating one or more aspects ofthe present invention.

DETAILED DESCRIPTION

As used herein, the terms “electronics rack”, “rack-mounted electronicequipment”, and “rack unit” are used interchangeably, and unlessotherwise specified include any housing, frame, rack, compartment, bladeserver system, etc., having one or more heat generating components of acomputer system or electronics system, and may be, for example, astand-alone computer processor having high, mid or low end processingcapability. In one embodiment, an electronics rack may comprise aportion of an electronic system, a single electronic system or multipleelectronic systems, for example, in one or more sub-housings, blades,books, drawers, nodes, compartments, etc., having one or moreheat-generating electronic components disposed therein. An electronicsystem(s) within an electronics rack may be movable or fixed relative tothe electronics rack, with rack-mounted electronic drawers and blades ofa blade center system being two examples of electronic systems (orsubsystems) of an electronics rack to be cooled.

“Electronic component” refers to any heat-generating electroniccomponent of, for example, a computer system or other electronic systemrequiring cooling. By way of example, an electronic component maycomprise one or more integrated circuit dies, and/or other electronicdevices to be cooled, such as one or more electronics cards comprising aplurality of memory modules (such as one or more dual in-line memorymodules (DIMMs)), or one or more data storage devices (e.g., replaceablehard drives). As used herein “primary heat-generating component” refersto a primary heat-generating electronic component (such as an integratedcircuit die) within an electronic system, while “secondaryheat-generating component” refers to an electronic component (such as areplaceable disk drive) generating less heat than the primaryheat-generating electronic component. By way of example, the primaryheat-generating electronic component may generate, in one embodiment,two times or more heat than the secondary heat-generating component.

Further, as used herein, the terms “liquid-cooled structure”,“liquid-cooled cold plate” and “liquid-cooled cold rail” refer tothermally conductive structures having one or more channels (orpassageways) formed therein or passing therethrough, which facilitatethe flow of liquid coolant through the structure. In one example, tubingis provided extending through the liquid-cooled cold plate orliquid-cooled cold rail. An “air-to-liquid heat exchanger” or“air-to-liquid heat exchange assembly” means any heat exchange mechanismcharacterized as described herein through which liquid coolant cancirculate; and includes, one or more discrete air-to-liquid heatexchangers coupled either in series or in parallel. An air-to-liquidheat exchanger may comprise, for example, one or more coolant flowpaths, formed of thermally conductive tubing (such as copper or othertubing) in thermal or mechanical contact with a plurality of air-cooledcooling fins. Size, configuration and construction of the air-to-liquidheat exchanger can vary without departing from the scope of theinvention disclosed. Still further, “coolant-cooled structure” refersto, for example, a liquid-cooled structure or a heat exchanger, such asan air-to-liquid heat exchanger. A coolant-cooled structure comprisesone or more coolant-carrying passages through which coolant circulatesto dissipate heat from an electronics rack. A “data center” refers to acomputer installation containing one or more electronics racks to becooled. As a specific example, a data center may comprise one or morerows of rack-mounted computer units, such as server units.

One example of coolant used within the cooled electronic apparatusesdisclosed herein is water. However, the concepts presented are readilyadapted to use with other types of coolant. For example, the coolant maycomprise a brine, a fluorocarbon liquid, a liquid metal, or othersimilar coolant, or refrigerant, while still maintaining the advantagesand unique features of the present invention.

Reference is made below to the drawings, which are not drawn to scalefor ease of understanding, wherein the same reference numbers usedthroughout different figures designate the same or similar components.

FIG. 1 depicts a raised floor layout of an air cooled data center 100typical in the prior art, wherein multiple electronics racks 110 aredisposed in one or more rows. A data center such as depicted in FIG. 1may house several hundred, or even several thousand microprocessors. Inthe arrangement illustrated, chilled air enters the computer room viaperforated floor tiles 160 from a supply air plenum 145 defined betweenthe raised floor 140 and a base or sub-floor 165 of the room. Cooled airis taken in through louvered covers at air inlet sides 120 of theelectronics racks and expelled through the back (i.e., air outlet sides130) of the electronics racks. Each electronics rack 110 may have one ormore air moving devices (e.g., fans or blowers) to provide forcedinlet-to-outlet airflow to cool the electronic devices within the rackunit. The supply air plenum 145 provides conditioned and cooled air tothe air-inlet sides of the electronics racks via perforated floor tiles160 disposed in a “cold” aisle of the computer installation. Theconditioned and cooled air is supplied to plenum 145 by one or more airconditioning units 150, also disposed within the data center 100. Roomair is taken into each air conditioning unit 150 near an upper portionthereof. This room air may comprise, in part, exhausted air from the“hot” aisles of the computer installation defined, for example, byopposing air outlet sides 130 of the electronics racks 110.

Due to ever-increasing air flow requirements through electronics racks,and the limits of air distribution within a typical data centerinstallation, liquid-based cooling is being combined with conventionalair-cooling. FIGS. 2-4 illustrate various embodiments of a data centerimplementation employing a liquid-based cooling system.

FIG. 2 depicts one rack-level liquid-cooling solution which utilizeschilled facility water to remove heat from the computer installationroom, thereby transferring the cooling burden from the air-conditioningunit(s) to the building's chilled water coolers. The embodiment depictedin FIG. 2 is described in detail in commonly assigned, U.S. Pat. No.6,775,137. Briefly summarized, facility-chilled water 200 circulatesthrough one or more liquid-to-liquid heat exchangers 210, coupled via asystem coolant loop 211, to individual electronics racks 220 within thecomputer room. Rack unit 220 includes one or more air-moving devices 230for moving air flow from an air inlet side to an air outlet side acrossone or more drawer units 240 containing heat-generating electroniccomponents to be cooled. In this embodiment, a front cover 250 attachedto the rack covers the air inlet side, a back cover 255 attached to therack covers the air outlet side, and a side car disposed adjacent to(and/or attached to) the rack includes a heat exchanger 260 for coolingair circulating through the rack unit. Further, in this embodiment, theliquid-to-liquid heat exchangers 210 are multiple computer roomwater-conditioning (CRWC) units which are coupled to receive buildingchilled facility water 200. The building chilled facility water is usedto cool the system coolant within system coolant loop 211, which iscirculating through air-to-liquid heat exchanger 260. The rack unit inthis example is assumed to comprise a substantially enclosed housing,wherein the same air circulates through the housing that passes acrossthe air-to-liquid heat exchanger 260. In this manner, heat generatedwithin the electronics rack is removed from the enclosed housing via thesystem coolant loop, and transferred to the facility coolant loop forremoval from the computer installation room.

FIG. 3 depicts another embodiment of a rack-level, liquid-coolingsolution, which again uses chilled facility water to remove heat fromthe computer installation room, thereby transferring the cooling burdenfrom the air-conditioning unit(s) to the building's chilled watercoolers. In this implementation, one embodiment of a coolantdistribution unit 300 for a data center is illustrated. Within coolantdistribution unit 300 is a power/control element 312, areservoir/expansion tank 313, a liquid-to-liquid heat exchanger 314, apump 315 (often accompanied by a redundant second pump), facility waterinlet 316 and outlet 317 supply pipes, a supply manifold 318 supplyingwater or system coolant to the electronics racks 110 via couplings 320and lines 322, and a return manifold 319 receiving water or systemcoolant from the electronics racks 110, via lines 323 and couplings 321.Each electronics rack includes (in one example) a power/control unit 330for the electronics rack, multiple electronic systems or subsystems 340,a system coolant supply manifold 350, and a system coolant returnmanifold 360. As shown, each electronics rack 110 is disposed on raisedfloor 140 of the data center with lines 322 providing system coolant tosystem coolant supply manifolds 350 and lines 323 facilitating return ofsystem coolant from system coolant return manifolds 360 being disposedin the supply air plenum beneath the raised floor.

In the embodiment illustrated, system coolant supply manifold 350provides system coolant to cooling apparatuses disposed within theelectronic systems or subsystems (for example, to liquid-cooled coldplates or cold rails) via flexible hose connections 351, which aredisposed between the supply manifold and the respective electronicsystems within the rack. Similarly, system coolant return manifold 360is coupled to the electronic systems via flexible hose connections 361.Quick connect couplings may be employed at the interface betweenflexible hoses 351, 361 and the individual electronic systems. By way ofexample, these quick connect couplings may comprise various types ofcommercially available couplings, such as those available from ColderProducts Company, of St. Paul, Minn., USA, or Parker Hannifin, ofCleveland, Ohio, USA.

Although not shown, electronics rack 110 may also include anair-to-liquid heat exchanger, for example, disposed at an air outletside thereof, which also receives system coolant from the system coolantsupply manifold 350 and returns system coolant to the system coolantreturn manifold 360.

FIG. 4 illustrates another embodiment of a liquid-cooled electronicsrack and cooling system therefor, in accordance with one or more aspectsof the present invention. In this embodiment, the electronics rack 400has a side car structure 410 associated therewith or attached thereto,which includes an air-to-liquid heat exchanger 415 through which aircirculates from an air outlet side of electronics rack 400 towards anair inlet side of electronics rack 400, for example, in a closed looppath in a manner similar to that illustrated above in connection withthe cooling implementation of FIG. 2. In this example, the coolingsystem comprises an economizer-based, warm-liquid coolant loop 420,which comprises multiple coolant tubes (or lines) connecting, in theexample depicted, air-to-liquid heat exchanger 415 in series fluidcommunication with a coolant supply manifold 430 associated withelectronics rack 400, and connecting in series fluid communication, acoolant return manifold 431 associated with electronics rack 400, acooling unit 440 of the cooling system, and air-to-liquid heat exchanger415.

As illustrated, coolant flowing through warm-liquid coolant loop 420,after circulating through air-to-liquid heat exchanger 415, flows viacoolant supply plenum 430 to one or more electronic systems ofelectronics rack 400, and in particular, one or more cold plates and/orcold rails 435 associated with the electronic systems, before returningvia coolant return manifold 431 to warm-liquid coolant loop 420, andsubsequently to cooling unit 440 disposed (for example) outdoors fromthe data center. In the embodiment illustrated, cooling unit 440includes a filter 441 for filtering the circulating liquid coolant, acondenser (or air-to-liquid heat exchanger) 442 for removing heat fromthe liquid coolant, and a pump 443 for returning the liquid coolantthrough warm-liquid coolant loop 420 to air-to-liquid heat exchanger415, and subsequently to the liquid-cooled electronics rack 400. By wayof example, hose barb fittings 450 and quick disconnect couplings 455may be employed to facilitate assembly or disassembly of warm-liquidcoolant loop 420.

As used herein, “warm liquid cooling” or “warm coolant cooling” refersto a cooling approach employing an outdoor-air-cooled heat exchange unitas the cooling unit 440. This heat exchange unit is coupled via, atleast in part, warm-liquid coolant loop 420 to dissipate heat from thecoolant passing through the cold plates and/or cold rails 435 associatedwith the electronic systems. In the embodiment depicted, the heat isdissipated from the coolant to outdoor ambient air. Thus, temperature ofthe coolant within warm-liquid coolant loop 420 is greater than thetemperature of the outdoor ambient air to which heat is dissipated. Byway of specific example, temperature of the coolant entering theliquid-cooled structures within the electronic system may be greaterthan or equal to 27° C. and less than or equal to 45° C. In one specificexample of the warm coolant-cooling approach of FIG. 4, ambienttemperature might be 30° C., and coolant temperature 35° C. leaving theair-to-liquid heat exchanger 442 of the cooling unit. The cooledelectronic system depicted thus facilitates a chiller-less data center.Advantageously, such a liquid-cooling solution provides highly energyefficient cooling of the electronic systems of the electronics rack,using liquid (e.g., water), that is cooled via circulation through theair-to-liquid heat exchanger located outdoors (i.e., a dry cooler) withexternal ambient air being pumped through the dry cooler. Note that thiswarm coolant-cooling approach of FIG. 4 is presented by way of exampleonly. In alternate approaches, cold coolant-cooling could be substitutedfor the cooling unit 440 depicted in FIG. 4. Such cold coolant-coolingmight employ building chilled facility coolant to cool the liquidcoolant flowing through the liquid-cooled electronics rack, andassociated air-to-liquid heat exchanger (if present), in a manner suchas described above in connection with FIGS. 2 & 3.

FIGS. 5A & 5B depict in greater detail one embodiment of a liquid-cooledelectronics rack, such as depicted in FIG. 4, in accordance with one ormore aspects of the present invention. In this implementation,liquid-cooled electronics rack 400 comprises a plurality of electronicsystems 500, within which one or more electronic components are to beliquid-cooled via, for example, one or more cold plates or cold rails,as described below. The cooling system includes coolant loop 420 coupledin fluid communication with coolant supply manifold 430 and coolantreturn manifold 431, both of which may comprise vertically-orientedmanifolds attached to liquid-cooled electronics rack 400. In thisembodiment, the rack-level coolant distribution system further includesindividual node-level supply hoses 510 supplying coolant from coolantsupply manifold 430 to cold plates and cold rails within the electronicsystems 500. As shown in FIG. 5B, coolant supply manifold 430 may be (inone embodiment) a vertically-oriented manifold with a plurality ofcoupling connections 511 disposed along the manifold, one for eachelectronic system 500 having one or more electronic components to beliquid-cooled. Coolant leaves the individual electronic systems 500 vianode-level return hoses 520, which couple the individual electronicsystems (or nodes) to coolant return manifold 431, and hence, to coolantloop 420. In the embodiment illustrated in FIG. 4, relativelywarm-liquid coolant, such as water, is supplied from the cooling unit,either directly, or through one or more air-to-liquid heat exchanger(s)415 (of FIG. 4), and hot coolant is returned via the coolant returnmanifold to the cooling unit. In one embodiment of the rack-levelcoolant distribution system illustrated in FIGS. 5A & 5B, the node-levelsupply and return hoses 510, 520 are flexible hoses.

FIG. 6 illustrates one embodiment of a cooled electronic system 500component layout, wherein one or more air-moving devices 600 provideforced air flow 601 to cool multiple components 610 within electronicsystem 500. Cool air is taken in through a front 602 and exhausted out aback 603 of the electronic system (or drawer). The multiple componentsto be cooled include, for example, multiple processor modules to whichliquid-cooled cold plates 620 (of the liquid-based cooling apparatus)are coupled, as well as multiple arrays 631, 632 of electronics cards630 (e.g., memory modules such as dual in-line memory modules (DIMMs)),which are to be thermally coupled to one or more liquid-cooled coldrails 625. As used herein “thermally coupled” refers to a physicalthermal transport path being established between components, forexample, between an electronics card and a liquid-cooled cold rail forthe conduction of heat from one to the other.

The illustrated liquid-based cooling approach further includes multiplecoolant-carrying tubes connecting in fluid communication liquid-cooledcold plates 620 and liquid-cooled cold rails 625. These coolant-carryingtubes comprise (for example), a coolant supply tube 640, multiple bridgetubes 641, and a coolant return tube 642. In the embodiment illustrated,bridge tubes 641 connect one liquid-cooled cold rail 625 in seriesbetween the two liquid-cooled cold plates 620, and connect in paralleltwo additional liquid-cooled cold rails 625 between the secondliquid-cooled cold plate 620 and the coolant return tube 642. Note thatthis configuration is provided by way of example only. The conceptsdisclosed herein may be readily adapted to use with variousconfigurations of cooled electronic system layouts. Note also, that asdepicted herein, the liquid-cooled cold rails are elongate, thermallyconductive structures comprising one or more channels through whichliquid coolant passes, for example, via one or more tubes extendingthrough the structures. The liquid-cooled cold rails are disposed, inthe embodiment illustrated, at the ends of the two arrays (or banks)631, 632 of electronics cards 630, and multiple thermal spreaders areprovided coupling in thermal communication electronics cards 630 andliquid-cooled cold rails 635.

FIGS. 7A-7D depict various alternate embodiments of a cooled electronicsystem, in accordance with one or more aspects of the present invention.These cooled electronic system comprise various embodiments of a coolingapparatus which include dynamic control and limiting of energy consumedby the cooling apparatus, as described hereinbelow.

Generally stated, disclosed herein is a cooling apparatus which includesone or more coolant-cooled structures, such as one or more liquid-cooledstructures (or cold plates) and/or one or more air-to-liquid heatexchangers, associated with an electronics rack for facilitatingdissipation of heat from the electronics rack in a manner, for example,such as described above in connection with FIG. 4. The one or morecoolant-cooled structures include one or more coolant-carrying passages.The cooling apparatus further includes a coolant loop coupled in fluidcommunication with the coolant-carrying passage(s) of the coolant-cooledstructure(s) and one or more heat exchange units coupled to facilitateheat transfer from coolant within the coolant loop. In one embodiment,the heat exchange unit(s) is coupled in fluid communication with thecoolant loop. The cooling apparatus further includes N controllablecomponents associated with at least one of the coolant loop or the atleast one heat exchange unit to facilitate at least one of circulatingof coolant through the coolant loop or transfer of heat from the coolantvia the heat exchange unit(s), wherein N≧1. Further, a controller iscoupled to the N controllable components. The controller dynamicallyadjusts operation of the N controllable components, based on Z inputparameters and one or more specified constraints, to provide a specifiedcooling to the at least one coolant-cooled structure, while limitingenergy consumed by the N controllable components, wherein Z≧1. In oneembodiment, the Z input parameters comprise Z measured input parameters,which may be dynamically-varying inputs or variables ascertained by thecontroller. By way of example, the input parameters may comprisetemperature readings, pressure readings, humidity readings, dew pointreadings, etc.

In the cooled electronic system embodiments of FIGS. 7A-7D, an air-side,economizer-based data center is depicted, which is implemented with acontrol process to limit data center cooling power consumption, inaccordance with one or more aspects of the present invention. Anair-side, economizer-based data center cooling system includes one ormore electronics racks which are partially or completely liquid-cooled,as well as, in one embodiment, a side car structure, an air-sideeconomizer (i.e., heat exchange unit), and one or more optionalliquid-to-liquid heat exchange buffers (depicted in FIGS. 7B-7D). In oneembodiment, substantially all the heat dissipated by the electroniccomponents within the electronics rack is transferred to the liquidcoolant; for example, partially within the electronics rack via one ormore liquid-cooled structures, and partially at the side car structure,via an air-to-liquid heat exchanger. This heat is then convected to theair-side economizer, where it is dissipated to the outdoor ambient air.The rate of heat transfer at the electronics rack(s) is predominantlygoverned by the coolant flow rate through the coolant-cooledstructure(s). At the economizer side, the heat transfer rate is governedby the economizer air-side flow rate and the liquid-coolant flow ratethrough the economizer. The heat transfer rate is a non-linear,monotonically increasing function of air-side flow rate and liquidcoolant flow rate.

For any given economizer design, there is a limit to the air-side flowrate and liquid coolant flow rate. These limits would typically be usedto guide the economizer selection to meet maximum cooling requirements(i.e., worst case scenario cooling requirements) by a safe margin. Asused herein, worst case scenario refers to the highest ambient airtemperature and highest heat dissipation at the electronics rack, and ina more general sense, the highest heat dissipation within the datacenter, occurring simultaneously. This situation is rare, and might noteven occur over the lifecycle of the data center. The typical result isa relatively high (more than required) cooling power consumption foralmost the entire lifecycle of the data center. Hence, disclosedhereinbelow are dynamic control processes, based on data center heatdissipation and, for example, ambient air temperature, which minimize(in one embodiment) the cooling power consumption, and thereby reducethe data center energy usage.

As noted, FIGS. 7A-7D depict alternate embodiments of a cooledelectronic system, in accordance with one or more aspects of the presentinvention, each comprising one or more electronics racks and a coolingapparatus such as described herein.

FIG. 7A depicts one embodiment of a single coolant loop configurationwhich includes one or more electronics racks 400, each with a side carstructure 410 associated therewith or attached thereto, which includesan air-to-liquid heat exchanger 415 through which air circulates from anair outlet side of electronics rack 400 towards an air inlet side ofelectronics rack 400, for example, in a closed loop path in a mannersimilar to that illustrated above in connection with the coolingimplementation of FIG. 2. In this example, the cooling system comprisesan economizer-based, warm-liquid coolant loop 700, which comprisesmultiple coolant tubes (or lines) connecting, in the example, depicted,air-to-liquid heat exchanger 415 in series fluid communication with acoolant supply manifold 430 associated with electronics rack 400, andconnecting in series fluid communication a coolant return manifold 431associated with electronics rack 400, and a coolant loop 700. By way ofexample, hose barb fittings 450 and quick disconnect couplings 455 maybe employed to facilitate assembly or disassembly of coolant loop 700.Coolant loop 700 includes a coolant pump 701 and is coupled in fluidcommunication with a heat exchange unit 710, which in the depictedembodiment, comprises a dry cooler, wherein ambient air 711 is forcedacross the heat exchange unit 710 via one or more fans 703. Anelectronically-controlled bypass valve 702 is coupled in parallel withheat exchange unit 710 to allow controlled bypassing of coolant withinthe coolant loop from passing through the heat exchange unit.

As illustrated, coolant flowing through warm-liquid coolant loop 700,after circulating through air-to-liquid heat exchanger 415, flows viacoolant supply plenum 430 to one or more electronic systems ofelectronics racks 400, and in particular, one or more liquid-cooled coldstructures 435 associated with the electronic systems, before returningvia coolant return manifold 431 to coolant loop 700, and subsequently,to heat exchange unit 710 disposed (for example) outdoors from a datacenter wall 705. In the embodiment illustrated, the heat exchange unitmay include a filter (not shown) for filtering the circulating liquidcoolant, as well as an air-to-liquid heat exchanger for removing heatfrom the liquid coolant. Note in the single-loop, cooled electronicsystem example of FIG. 7A, air-to-liquid heat exchanger 415 andliquid-cooled cold structures 435 are examples of coolant-cooledstructures associated with the electronics rack for facilitatingdissipation of heat from the electronics rack. Additionally,controllable components associated with at least one of the coolant loop700 or the at least one heat exchange unit 710 include coolant pump 701,fan 703, and recirculation (or bypass) valve 702.

Note that as used herein, “warm-liquid cooling” or “warm-coolantcooling” refer to a cooling approach employing an outdoor-air-cooledheat exchange unit. This heat exchange unit is coupled via, at least inpart, coolant loop 700 to dissipate heat from the coolant passingthrough the coolant-cooled structures 435, 415 associated with theelectronic systems. In the embodiment depicted, this heat is dissipatedfrom the coolant to outdoor ambient air. Thus, temperature of thecoolant within warm-liquid coolant loop 700 is greater than thetemperature of the outdoor ambient air to which heat is dissipated. Byway of specific example, temperature of coolant entering theliquid-cooled structures within the electronic systems may be greaterthan or equal to 27° C., and less than or equal to 45° C. The cooledelectronic system depicted in FIG. 7A thus facilitates a chiller-lessdata center. Advantageously, such a liquid-cooling solution provideshighly energy efficient cooling of the electronic systems of theelectronics rack, using liquid (e.g., water) that is cooled viacirculation through the heat exchange unit(s) 710 located outdoors(i.e., a dry cooler) with external ambient air 711 being pumped throughthe dry cooler. Note that this single loop, warm-cooling approach ofFIG. 7A is presented by way of example, only. Several alternativeapproaches are depicted in FIGS. 7B-7D, and described below.

As illustrated in FIG. 7A, a controller 720 is also provided as part ofthe cooling apparatus. For the single loop configuration of FIG. 7A,input parameters to controller 720 may include ambient temperature andhumidity 721, temperature of one or more electronic components 722, suchas processor, memory or hard drive components, a dew point sensor 723within the data center, and temperature and pressure 724 of coolantwithin the coolant loop 700, for example, temperature and pressure ofcoolant being provided to the air-to-liquid heat exchanger 415 of theside car structure 410 in the embodiment illustrated in FIG. 7A. Asnoted, in this embodiment, the N controllable components include coolantpump(s) 701, fans(s) 703, and electronically-controlled bypass valve702. Controller 720 provides a corresponding operational setting to eachof these components as described herein via control lines 730, 731, 732,respectively. In one example, the operational control setting forcoolant pump(s) 701 might comprise an RPM value, the operational settingfor fan(s) 703 might comprise a different RPM value, and an operationalsetting for bypass valve 702 might comprise a percent (%) openparameter, which controls the positioning of the valve. Depending on thecontrol process implemented by controller 720, other input parameters,or other output operational control settings, may be employed orprovided.

FIG. 7B depicts a dual-loop cooling apparatus embodiment, which issimilar to the single loop embodiment described above in connection withFIG. 7A, except that a liquid-to-liquid heat exchanger 740 is providedwithin the data center as a buffer to coolant flowing through coolantloop 700 and a secondary coolant flowing through heat exchange unit 710.This advantageously allows, for example, water to be employed withincoolant loop 700, and a different coolant (for example, a water glycolmixture) to be employed within secondary coolant loop 704 coupling influid communication liquid-to-liquid heat exchanger 740 and heatexchange unit 710. Additional components for the cooling apparatus ofFIG. 7B include a second pump 741 associated with secondary coolant loop704, coupling in fluid communication liquid-to-liquid heat exchanger 740and heat exchange unit 710, and an additional temperature and pressuresensor 725 for sensing temperature and pressure of coolant withinsecondary coolant loop 704. In addition, controller 720 of the coolingapparatus of FIG. 7B provides, via a control line 733, an outputoperational control setting for the second coolant pump 741 in additionto those operational control settings provided in connection with theembodiment of FIG. 7A.

FIG. 7C depicts a further cooling apparatus embodiment, wherein multipleheat exchangers 740 are coupled in series between coolant loop 700 andheat exchange unit 710. These heat exchangers may each comprise aliquid-to-liquid heat exchanger, a liquid-to-gas heat exchanger, or evena gas-to-gas heat exchanger. Additionally, the cooling apparatus of FIG.7C includes additional cooling pumps 751, each associated with arespective coolant loop 750 coupling two coolant-to-coolant heatexchangers 740. Controller 720 includes similar temperature and pressuresensors for determining temperature and pressure of coolant within eachcoolant loop 700, 750, 704, and outputs operational control settings forthe additional controllable components comprising coolant pumps 751, inaddition to the above-discussed controllable components.

FIG. 7D depicts an alternate cooling apparatus embodiment, wherein theadditional coolant loops are coupled in parallel rather than in series,as depicted in FIG. 7C. In this embodiment, coolant loop 700 is coupledvia multiple parallel-connected, coolant-to-coolant heat exchangers 740to multiple heat exchange units 710 via respective secondary coolantloops 704, and secondary coolant pumps 741. In addition to coolant pumps701, 741, coolant pumps 751 are provided, if desired, for supplementalcoolant pumping through the parallel-connected, coolant-to-coolant heatexchangers 740. In the embodiment illustrated in FIG. 7D, each secondarycoolant loop 704 includes a respective coolant pump 741, and arespective heat exchange unit 710, as well as one or more fans 703associated with the heat exchange unit, and an electronically-controlledbypass valve 702 for selectively bypassing a portion of the secondarycoolant from the heat exchange unit 710. Quick connect couplings 760 maybe employed to facilitate assembly or disassembly of the coolingapparatus illustrated.

Note that FIGS. 7C & 7D depict cooling apparatuses wherein the RPMs of aplurality of economizer fans, and a plurality of coolant pumps areregulated individually, or simultaneously, to limit or optimize datacooling energy consumption, and subsequently, to reduce the total datacenter energy consumption.

Proposed herein are various control processes for, for example, anair-side, economizer-based data center cooling approach such as depictedin FIGS. 7A-7D, which can be implemented to optimize or limit powerconsumption of the data center cooling. The control process embodimentspresented herein (by way of example only) are based on regulating, forexample, the economizer fan RPMs, as well as the coolant pump(s) RPMs,and in certain embodiments, one or more flow control valve settings.These operational settings are regulated based on Z input parameters,which may include various input conditions or parameters, such asoutdoor ambient air temperature and humidity, data center temperatureand humidity, temperature of one or more electronic components withinthe electronics rack being cooled, temperature and pressure of one ormore coolants within the cooling apparatus, flow rates through one ormore cooling loops, as well as a dew point of the data center. Forexample, if one or more economizer fans are running at maximum RPMs for40° C. ambient air, and a maximum heat load dissipation at theelectronics rack, they may run at a lower RPM (for example, 50% of maxRPMs), at a lower ambient air temperature (for example, 20° C.). Thiscan reduce the fan power consumption by 87.5% of maximum. Note that thepower consumption for a fan, as well as for a coolant pump, follows acubic relationship with the corresponding operational RPMs. In otherwords, by reducing the fan's RPMs by 50%, fan power consumption reducesby 87.5%. In a more general sense, RPMs of N number of economizer fans,and M number of coolant pumps, can be dynamically regulatedsimultaneously to reduce or limit the data center cooling energy, andsubsequently reduce the total data center energy consumption.

FIGS. 8A-8C depict one embodiment of a control process which can beemployed to minimize data center cooling power consumption, inaccordance with one or more aspects of the present invention. Thiscontrol process comprises a data structure or table-based controlapproach, as explained below.

Control processing begins with a selected, prespecified data structure(or table) being provided to the control process 800, one embodiment ofwhich is depicted in FIG. 8B. Controllable components such as fans andcoolant pumps are initially set with a maximum RPM, and anyrecirculation valves are fully closed 805. By design, all specifiedconstraints for the operation of the cooling apparatus should besatisfied 810. By way of example, determining whether constraints aremet might comprise determining whether temperature of an electroniccomponent, such as a processor, is less than a specified temperature(T_(CPU)<T_(CPU,spec)?), determining whether temperature of anassociated memory module, such as a dual in-line memory module, is lessthan a specified, dual in-line memory module temperature(T_(DIMM)<T_(DIMM,spec)?), and/or determining whether air inlettemperature to the electronics rack is less than a specified air inlettemperature to the electronics rack(T_(rack air in)<T_(rack air in,spec)?). If “no”, then a warning messageof insufficient cooling is displayed, and the controller maintains thecontrollable components at their maximum operational control settings815.

If the specified constraints are satisfied, then the Z input parametersfor the selected, specified table are checked to ascertain a criticalconstraint 820. As illustrated in FIG. 8B, the table may comprise “r”number of rows, and “Z” number of input parameters (or conditions). Theinput parameters (or conditions) could be any or all of the monitoredvariables, such as the temperatures, pressures, humidity, flow rates,and power consumption at various locations within the cooled electronicsystem. The output conditions in the table are the operational controlsettings for, for example, fans and coolant pumps, as well as anyrecirculation valve positions. In case of a single input condition,there is no need to identify the critical constraint, and the valvesetting(s) and the output condition(s) can be assigned defined in thetable.

For the case of two or more input conditions, the critical constraint isidentified before applying values of the output conditions. Based on theactual value of the monitored input parameters, each input can beassigned a row number. The input condition having the highest row numberis, in one embodiment, the critical constraint. For example, let therebe two input conditions, I1 and 12, and let their actual monitoredvalues be I1,x and I2,x. With the selected table, the value I1,x couldbe between the table values of I1,r 1 and I1,r 2 (where row r1<r2),while the value I2,x could be between the values of I2,r 3 and I2,r 4(where row r3<r4). If input I1 were selected as the constraint, thecorresponding row number would be rl, whereas if I2 is selected as theconstraint, the corresponding row number would be r3. If r3>r1, theninput 12 would be the critical condition, otherwise, input Il would bethe critical constraint. If r1=r3, then both the inputs are criticalconstraints. Once the critical constraints are identified, the set ofoperational control settings (or output conditions to be applied) areidentified from the table. FIG. 8C depicts one embodiment of thisprocess.

As illustrated in FIG. 8C, the controller obtains a value for each inputparameter (or condition) in the selected table 825, and associates a rownumber with each input parameter using the obtained values of the inputparameters 830. If the value of the obtained input parameter liesbetween two table values, then the lower row number is employed. Thecontroller then chooses the highest row number by identifying thecritical constraint(s), i.e., in one embodiment, the input parametervalue(s) having the highest row number 835. The corresponding set ofoperational control settings (or output conditions) in this row of theplurality of rows in the specified table, are then applied, based on thehighest row number identified 840.

Referring to FIG. 8A, the health/operating conditions of the cooledelectronic system may next be reported 845. For example, temperature,pressure, humidity, flow rates, power consumption, valve positions,etc., can be reported or displayed to reflect the outcome of the appliedset of operational control settings (or output conditions). Thecontroller may check if maintenance is required 850. For example,maintenance may be required based on hours of system operation, or anysimilar condition. If “yes”, then a “maintenance required” message maybe displayed, and processing may switch to maximum operational controlsettings 855. If “no”, then processing continues in normal operation,which may include a check for whether a manual override has been made860. If there is a manual override, then the system will switch tomanual mode settings 865, and continue to run in the manual mode untiluser input is provided to reenter the control loop 870. If there is nomanual override, then the system will continue normal operation, and thecontroller will, after waiting a time interval t 825, return to checkthe current input parameters on the selected table 820.

Note that the table values comprising the input parameters and sets ofoperational control settings may be determined through experimentaltesting in a lab environment. The loop configuration can be initiallyinstalled such that the dry cooler receives a supply of air at acontrolled temperature. This can be carried out using an air heaterapparatus upstream of the dry cooler, and by initially having the drycooler located indoors to facilitate testing before transferring to anoutdoor environment. The air heater apparatus can comprise anelectrical, resistance-based heater, that is conductively coupled tometal heat sinks and fans being used to force air through the heatedheat sinks Alternatively, an air-to-liquid heat exchanger with a waterheater fluidically coupled to the water loop of the heat exchanger couldbe used as the air heater apparatus. Some level of heated air ductingfrom the air heated apparatus to the inlet of the air side of the drycooler heat exchanger coils may be required.

Once a controllable air heater apparatus is in place, the system can bestarted, that is, the coolant pumps and fans and the IT load, andfinally the air heater apparatus, may be activated. The coolant pumpsand fans can be set to their maximum speeds, and the IT load can beramped up slowly to maximum load condition. After steady state isreached (i.e., after the temperatures stabilize), air temperatures atthe inlet to the dry cooler heat exchanger coils can be measured andcompared to the highest temperature expected annually, based on localweather data. If the inlet air temperature to the dry cooler coil isless than the maximum air temperature expected based on local weatherconditions (as would be likely for most of the year), the heaters areswitched ON and controlled until the maximum air inlet temperaturecondition at the inlet to the dry cooler is met. The cooling apparatusis run at these conditions until steady state is reached.

After steady state is reached, the speeds of the various coolant pumpsand fans are ramped down slowly in small increments from their maximumspeed. While reducing the fan and pump speeds, the speed of each coolantpumping device can be reduced in small steps that are related to eachother, that is, using a factor, or the speeds can be reducedindividually using a fixed percentage change between the maximum andminimum allowable pump or fan speeds. After each decrement of the pumpand fan speeds, steady state is allowed to be reached and various deviceand coolant temperatures of interest are recorded.

The table generated using the experiments described above then providesa guiding relationship between the coolant and device temperatures andthe fan and pump speeds, while assuming a maximum IT load and a maximuminlet air temperature to the dry cooler heat exchanger coil. The use ofthe maximum IT load and dry cooler air inlet temperatures results in aconservative control of the system with respect to the pump and fan flowrates. However, cooling energy efficiency is achieved using the rationalreduction of pump and fan speeds for lower coolant or devicetemperatures. Further, experiments can be carried out to parametricallytest various factors that relate the fan and pump speeds to each other.Alternatively, various individual profiles for decreasing the individualcoolant pumps and fans could be tested. After a matrix of tests has beenperformed, the most energy efficient (i.e., best performing) parameterinputs to the table can be determined and chosen. If seasonalsensitivity is desired, a typical day for different seasons (i.e., fall,winter, spring and summer) can be created using the controllable airheater apparatus and data obtained as the equipment speeds are variedusing the methods outlined to generate a season-specific table. Aftersatisfactory experimentation, the dry cooler can be commissioned and thecoolant loop can be allowed to become fully operational.

FIGS. 9A & 9B depict an alternate control process, wherein thecontrollable components are initiated at maximum operational controlsettings, and control setting changes are made to all controllablecomponents in a related manner. Specifically, control processing isinitiated 900 by setting the controllable components to maximumoperational control settings, and any recirculation valves fully closed905. By design, at this point, the constraint(s) should be satisfied910, and if “no”, processing determines whether the fans and coolantpumps are running at their maximum operational state and anyrecirculation valves are fully closed 915. If “yes”, then aninsufficient cooling warning is issued, and the controllable componentscontinue to operate at their maximum operational control settings 920.If the fans and coolant pumps are not running at their maximumoperational control settings 915, then processing sets a “flag” variableequal to 2 940, which indicates that operational control settings are tobe increased, as explained further below. If the specified constraintsare met 910, then processing checks the margins by which each inputparameter meets its associated constraint, and determines whether theminimum margin of the ascertained margins is greater than a minimumspecified margin 925. Note, in this regard, that margins (M1, M2 . . . )can be normalized to vary from 1 to 100, and that Z is (in one example)also the number of constraints. If all of the ascertained margins aregreater than the minimum specified margin, then the variable “flag” isassigned a value 1 945, which is a reduce cooling power consumptionmode, as explained below. If all the margins are not greater than theminimum specified margin, then processing checks to determine whetherthe fans and coolant pumps are running at their respective maximumoperational control settings, and if all the recirculation, bypassvalves are fully closed. If all the controllable components are runningat their maximum state, then the system will continue to run at thatstate 935. Otherwise, processing sets the “flag” variable to a value 2.

As illustrated in FIG. 9B, processing next determines whether the “flag”variable equals 1 950, and if “yes”, determines whether the controllablecomponents (or cooling devices) are all running at their minimumoperational control settings 955. If “no”, then processing reduces allcontrollable components by a corresponding ΔRPM, or opens therecirculation valves by a corresponding Δ% 960, before returning to theprocess of FIG. 9A to determine whether maintenance is required 975.

Continuing with FIG. 9B, if the “flag” variable is other than 1,processing determines whether the controllable components (or coolingdevices) are running at their maximum operational control settings 965,and if “no”, increases RPMs of all controllable components by ΔRPMs, orcloses all recirculation valve(s) by Δ% 970, after which processingreturns to the control process of FIG. 9A to determine whethermaintenance is required 975. In one embodiment, the entire ΔRPM and Δ%increases or decreases are related. For example, these relations mightcomprise:

ΔRPM_(—) F2=X2*ΔRPM_(—) F1, ΔRPM_(—) F3=X3*ΔRPM_(—) F1, . . . , ΔRPM_(—)FN=XN*ΔRPM_(—) F1   (1)

ΔRPM_(—) P1=Y1*ΔRPM_(—) F1, ΔRPM_(—) P2−Y2*ΔRMP_(—) F1, . . . , ΔRPM_(—)P2N=Y2N*ΔRPM_(—) F1   (2)

Δ% Open_(—) V1=W1*ΔRPM_(—) F1, Δ% Open_(—) V2=W2*ΔRPM_(—) F1, . . . , Δ%Open_(—) VN=WN*ΔRPM_(—) F1   (3)

where ΔRPM_F1 is the change in RPM for fan F1, ΔRPM_F2 is the change inRPM to fan F2, etc.

Continuing with FIG. 9A, processing checks whether maintenance isrequired 975, and if “yes”, switches the controllable components tomaximum operational control settings and displays a “maintenancerequired” indication 980. If maintenance is not required, thenprocessing determines whether there has been a manual override 985, andif “yes”, switches to manual mode settings, and displays a “manual mode”indication 990. Thereafter, processing waits a time interval t 995before again checking whether the constraints have been met by thecurrent input parameters 910.

FIGS. 10A & 10B depict another control process, in accordance with oneor more aspects of the present invention, wherein the controllablecomponents are initiated at maximum operational control settings, andcontrol setting changes are made one component at a time.

Specifically, control processing is initiated 1000 by setting thecontrollable components to maximum operational control settings, and anyrecirculation bypass valves fully closed 1002. By design, at this point,the constraint(s) should be satisfied 1004, and if “no”, then processingdetermines whether the fans and coolant pumps are running at theirmaximum operational state and any recirculation valves are fully closed1006. If “yes”, then an insufficient cooling warning is issued, and thecontrollable components continue to operate at their maximum controlsettings 1008. If the fans and coolant pumps are not running at theirmaximum operational control settings 1006, then processing sets a “flag”variable equal to 2 1016, which indicates that operational controlsettings are to be serially, incrementally increased, as explainedfurther below. If the specified constraints are met 1004, thenprocessing checks the margins by which each input parameter meets itsassociated constraint, and determines whether the minimum margin of theascertained margins is greater than a minimum specified margin 1010. Ifall of the ascertained margins are greater than the minimum specifiedmargin, then the “flag” variable is assigned a value 1 1018, indicativeof a reduce cooling power consumption mode, as explained below. If allthe margins are not greater than the minimum specified margin, thenprocessing checks to determine whether the fans and coolant pumps arerunning at their respective maximum operational control settings, andall of the recirculation valves are fully closed 1012. If all thecontrollable components are running at their maximum state, thenprocessing will continue to run at that state 1014. Otherwise,processing sets the “flag” variable to a value 2 1016.

As illustrated in FIG. 10B, processing next determines whether the“flag” variable equals 1 1020 (that is, is the flag set to indicate areduce cooling power consumption mode). If “yes”, then the states of thecontrollable components, e.g., fans and coolant pumps, are sequentiallychecked, for example, in descending order, starting with fan FN,followed by fan FN-1, and so on, until reaching pump P1. Specifically,in the embodiment illustrated, the state of the N^(th) fan (that is, fanFN) is initially checked 1022. If fan FN is not running at its minimumoperational control setting, then fan FN is reduced by ΔRPM_FN 1024, andcontrol processing stores/remembers the latest state change by settingan “undo_flag” equal to zero 1026, before returning to the processing ofFIG. 10A. If fan FN is running at its minimum operational control state,then processing checks whether fan FN-1 is running at its minimumoperational control state 1028, and if “no”, reduces fan FN-1 speed byΔRPM_FN-1 1030. Processing repeats the inquiries until reaching fan FN1to determine whether it is running at its minimum operational controlstate 1032, and if “no”, reduces speed of fan FN1 by ΔRPM_F1 1034. Onceall fan speeds have been reduced, further power consumption savings isobtained by evaluating the coolant pumps and serially, incrementallyreducing coolant pump operation, if possible. Specifically, pump P2N isevaluated to determine whether it is running at minimum operationalcontrol setting 1036, and if “no”, speed of pump P2N is reduced byΔRPM_P2N 1038. Once pump P2N is running at its minimum operationalcontrol setting, then processing evaluates pump P2N-1 to determinewhether it is running at its minimum operational control state 1040, andif “no”, reduces speed of pump P2N-1 by ΔRPM_P2N-1 1042. The processcontinues until pump P1 is reached and a determination is made whetherpump P1 is running at its minimum operational control state 1044, and if“no”, processing reduces speed of pump P1 RPMs by ΔRPM_P1 1046. If allcontrollable components are running at minimum operational controlsettings, then processing returns to the flow of FIG. 10A, whereindicated.

If the value of “flag” variable is other than 1, that is, it is 2,indicative of an increase cooling mode, processing initially checks thevalue of the “undo_flag” variable 1050 to see whether it equals 1. If“no”, then processing undoes the last operational control setting change1052, and returns to the processing of FIG. 10A. Assuming that the“undo_flag” variable is 1, then the states of the cooling devices, thatis, the fans and pumps, are checked in ascending order, starting frompump P1, followed by pump P2, and so on, until fan FN. Note that in thisembodiment, the ordering of the fans and pumps may be any desired order,but that the increasing of operational control settings would be in anopposite order from the decreasing of operational control settingsdescribed above.

By way of example, power consumption may be initially minimized byevaluating the highest-power-consuming components lost. In the examplepresented, the state of pump P1 is initially checked to determinewhether pump P1 is running at its maximum operational control setting1054, and if “no”, then the speed of pump P1 is increased by ΔRPM_P11056. Once pump P1 is running at its maximum operational controlsetting, pump P2 is evaluated to determine whether it is running atmaximum operational control setting 1058, and if “no”, the speed pump P2is increased by ΔRPM_P2 1060. This process is repeated until reachingpump P2N, and determining whether it is running at maximum operationalcontrol setting 1062, and if “no”, then speed of pump P2N is increasedby ΔRPM_P2N 1064.

After all coolant pumps are running at maximum operational controlsetting, then the fans are evaluated (in this example). Specifically,processing determines whether fan F1 is running at maximum operationalcontrol setting 1066, and if “no”, increases speed of fan F1 by ΔRPM_F11068. Once fan F1 is running at maximum operational control setting, andassuming further cooling is required, processing determines whether fanF2 is running at maximum operational control setting 1070, and if “no”,increases speed of fan F2 by ΔRPM_F2 1072. This process is repeateduntil processing reaches fan FN, and determines whether fan FN isrunning at maximum operational control setting 1074, and if “no”,increases speed of fan FN by ΔRPM_FN 1076, after which, processingreturns to FIG. 10A where indicated.

Continuing with FIG. 10A, processing checks whether maintenance isrequired 1080, and if “yes”, switches the controllable components tomaximum operational control settings and displays a “maintenancerequired” indication 1082. If maintenance is not required, thenprocessing determines whether there has been a manual override 1084, andif “yes”, switches to manual mode settings, and displays a “manual mode”indication 1086. Otherwise, processing waits a time interval t 1088before again checking whether the constraints have been met by thecurrent input parameters 1004.

FIGS. 11A-11D depict another control process, in accordance with one ormore aspects of the present invention, wherein the controllablecomponents are initiated at maximum operational control settings, andcontrol setting changes are made one at a time. This embodiment issimilar to that described above in connection with FIGS. 10A-10B, exceptprovision is made for also dynamically adjusting any recirculationvalves within the cooling apparatus.

Beginning with FIG. 11A, control processing is initiated 1100 by settingthe controllable components to maximum operational control settings, andthe recirculation bypass valves fully closed 1102. By design, at thispoint, the constraint(s) should be satisfied 1104, and if “no”, thenprocessing determines whether the fans and coolant pumps are running attheir maximum operational state and any recirculation valves are fullyclosed 1106. If “yes”, then an insufficient cooling warning is issued,and the controllable components continue to operate at their maximumcontrol settings 1108. If the fans and coolant pumps are not running attheir maximum operational control settings 1106, then processing sets a“flag” variable equal to 2 1116, which indicates that operationalcontrol settings are to be serially, incrementally increased, asexplained further below. If the specified constraints are met 1104, thenprocessing checks the margins by which each input parameter meets itsassociated constraint, and determines whether the minimum margin of theascertained margins is greater than a minimum specified margin 1110. Ifall of the ascertained margins are greater than the minimum specifiedmargin, then the “flag” variable is assigned a value 1 1118, indicativeof a reduce cooling power consumption mode, as explained below. If allthe margins are not greater than the minimum specified margin, thenprocessing checks to determine whether the fans and coolant pumps arerunning at their respective maximum operational control settings, andall of the recirculation valves are fully closed 1112. If all thecontrollable components are running at their maximum state, thenprocessing will continue to run at that state 1114. Otherwise,processing sets the “flag” variable to a value 2 1116.

As illustrated in FIG. 11B, processing next determines whether the“flag” variable equals 1 1120 (that is, is the flag set to indicate areduce cooling power consumption mode). If “yes”, then the states of thecontrollable components, e.g., fans and coolant pumps, are sequentiallychecked, for example, in descending order, starting with fan FN,followed by fan FN-1, and so on, until reaching pump P1. Specifically,in the embodiment illustrated, the state of the N^(th) fan (that is, fanFN) is initially checked 1122. If fan FN is not running at its minimumoperational control setting, then fan FN is reduced by ΔRPM_FN 1124, andcontrol processing stores/remembers the latest state change by settingan “undo_flag” equal to zero 1126, before returning to the processing ofFIG. 11A. If fan FN is running at its minimum operational control state,then processing checks whether fan FN-1 is running at its minimumoperational control state 1128, and if “no”, reduces fan FN-1 speed byΔRPM_FN-1 1130. Processing repeats the inquiries until reaching fan FN1to determine whether it is running at its minimum operational controlstate 1132, and if “no”, reduces fan FN1 speed by ΔRPM_F1 1134. Once allfan speeds have been reduced, further power consumption savings may beobtained by evaluating the coolant pumps and serially, incrementallyreducing coolant pump operation, if possible. Specifically, pump P2N isevaluated to determine whether it is running at minimum operationalcontrol setting 1136, and if “no”, speed of pump P2N is reduced byΔRPM_P2N 1138. Once pump P2N is running at its minimum operationalcontrol setting, then processing evaluates pump P2N-1 to determinewhether it is running at its minimum operational control state 1140, andif “no”, reduces speed of pump P2N-1 by ΔRPM_P2N-1 1142. The processcontinues until pump P1 is reached and a determination is made whetherpump P1 is running at its minimum operational control state 1144, and if“no”, processing reduces speed of pump P1 by ΔRPM_P1 1146.

Once all pumps are running at minimum operational control settings, thenprocessing passes to the flow of FIG. 11C, and determines whetherrecirculation valve VN is fully open 1145, and if “no”, further opensthe valve VN by Δ%_VN 1147, before returning to FIG. 11B, tostore/remember the latest change my setting the “undo_flag” equal tozero, and then returning to the process of FIG. 11A, wherein indicated.Once recirculation valve VN is fully open, then processing determineswhether recirculation valve VN-1 is fully open 1149, and if “no”,incrementally opens valve VN-1 by Δ%_VN-1 1151. This one at a time,incremental adjusting of the valves repeats until processing reachesvalve V1, and determines whether recirculation valve V1 is fully open1153, and if “no”, opens valve V1 by Δ%_V1 1155. Otherwise, if allcontrollable components have been adjusted to conserve cooling energy,processing returns to the flow of FIG. 11A, where indicated.

Continuing with FIG. 11B, if the value of “flag” variable is other than1, that is, it is 2, indicative of an increase cooling mode, processinginitially checks the value of the “undo_flag” variable 1150 to seewhether it equals 1. If “no”, then processing undoes the lastoperational control setting change 1152, and returns to the processingof FIG. 11A, where indicated. Assuming that the “undo_flag” variable is1, then the states of all cooling devices, that is, the fans, pumps, andrecirculation valves are checked in ascending order, starting fromrecirculation valve V1 (FIG. 11D), followed by recirculation valve V2,and so on, until reaching fan FN. Note that in this embodiment, theordering of the fans, pumps and valves may be any desired ordering, butthat the increasing of operational control settings is in an oppositeorder from the decreasing of operational control settings describedabove.

As noted, FIG. 11D evaluates and adjusts the recirculation valves todetermine the incremental closures to be made in the increase coolingmode. Processing initially determines whether recirculation valve V1 isfully closed 1161, and if “no”, incrementally closes recirculation valveV1 by Δ%_V1 1163, before returning to the processing of FIG. 11A, whereindicated. Once valve V1 is fully closed, processing determines whetherrecirculation valve V2 is fully closed 1165, and if “no”, incrementallycloses valve V2 by Δ%_V2 1167. The incremental valve closures continueuntil valve VN is reached, and processing determines whetherrecirculation valve VN is fully closed 1169, and if “no”, incrementallycloses valve VN by Δ%_VN 1171. Once all recirculation valves are fullyclosed, processing continues with the control process of FIG. 11B, whereindicated.

As illustrated in FIG. 11B, additional cooling is provided, in thisexample, by checking the state of pump P1 to determine whether pump P1is running at maximum operational control setting 1154, and if “no”,then the speed of pump P1 is increased by ΔRPM_P1 1156. Once pump P1 isrunning at maximum operational control setting, pump P2 is evaluated todetermine whether it is running at maximum operational control setting1158, and if “no”, then the speed of pump P2 is increased by ΔRPM_P21160. This process is repeated until reaching pump P2N, and determiningwhether it is running at maximum operational control setting 1162, andif “no”, then the speed of pump P2N is increased by ΔRPM_P2N 1164.

After all coolant pumps are running at maximum operational controlsetting, then the fans are evaluated (in this example). Specifically,processing determines whether fan F1 is running at maximum operationalcontrol setting 1166, and if “no”, increases the speed of fan F1 byΔRPM_F1 1168. Once fan F1 is running at maximum operational controlsetting, processing determines whether fan F2 is running at maximumoperational control setting 1170, and if “no”, increases the speed offan F2 by ΔRPM_F2 1172. This process is repeated until processingreaches fan FN, and determines whether fan FN is running at maximumoperational control setting 1174, and if “no”, increases the speed offan FN by ΔRPM_FN 1176, after which, processing returns to FIG. 11A,where indicated.

Continuing with FIG. 11A, processing checks whether maintenance isrequired 1180, and if “yes”, switches the controllable components tomaximum operational control settings and displays a “maintenancerequired” indication 1182. If maintenance is not required, thenprocessing determines whether there has been a manual override 1184, andif “yes”, switches to manual mode settings, and displays a “manual mode”indication 1186. Otherwise, processing waits a time interval t 1188before again checking whether the constraints have been met by thecurrent input parameters 1104.

Note again that the incremental, stepwise processing of FIGS. 11A-11Dproceeds in a number-based manner. The controllable components, i.e.,recirculation valves, pumps and fans (in this example), may be numberedin any desired manner. For example, the components may be numbered basedon physical location within the cooling system, or based on power thatthe components consume at their maximum state, with the highest numbergiven to the unit that consumes the maximum power, and the lowest numbergiven to the unit that consumes the minimum power.

As will be appreciated by one skilled in the art, control aspects of thepresent invention may be embodied as a system, method or computerprogram product. Accordingly, aspects of the present invention may takethe form of an entirely hardware embodiment, an entirely softwareembodiment (including firmware, resident software, micro-code, etc.) oran embodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system”.Furthermore, control aspects of the present invention may take the formof a computer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readable signalmedium may be any non-transitory computer readable medium that is not acomputer readable storage medium and that can communicate, propagate, ortransport a program for use by or in connection with an instructionexecution system, apparatus or device.

A computer readable storage medium may be, for example, but not limitedto, an electronic, magnetic, optical, electromagnetic, infrared orsemiconductor system, apparatus, or device, or any suitable combinationof the foregoing. More specific examples (a non-exhaustive list) of thecomputer readable storage medium include the following: an electricalconnection having one or more wires, a portable computer diskette, ahard disk, a random access memory (RAM), a read-only memory (ROM), anerasable programmable read-only memory (EPROM or Flash memory), anoptical fiber, a portable compact disc read-only memory (CD-ROM), anoptical storage device, a magnetic storage device, or any suitablecombination of the foregoing. In the context of this document, acomputer readable storage medium may be any tangible medium that cancontain or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

Referring now to FIG. 12, in one example, a computer program product1200 includes, for instance, one or more computer readable storage media1202 to store computer readable program code means or logic 1204 thereonto provide and facilitate one or more aspects of the present invention.

Program code embodied on a computer readable medium may be transmittedusing an appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programminglanguage, such as Java, Smalltalk, C++ or the like, and conventionalprocedural programming languages, such as the “C” programming language,assembler or similar programming languages. The program code may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider).

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

In addition to the above, one or more aspects of the present inventionmay be provided, offered, deployed, managed, serviced, etc. by a serviceprovider who offers management of customer environments. For instance,the service provider can create, maintain, support, etc. computer codeand/or a computer infrastructure that performs one or more aspects ofthe present invention for one or more customers. In return, the serviceprovider may receive payment from the customer under a subscriptionand/or fee agreement, as examples. Additionally or alternatively, theservice provider may receive payment from the sale of advertisingcontent to one or more third parties.

In one aspect of the present invention, an application may be deployedfor performing one or more aspects of the present invention. As oneexample, the deploying of an application comprises providing computerinfrastructure operable to perform one or more aspects of the presentinvention.

As a further aspect of the present invention, a computing infrastructuremay be deployed comprising integrating computer readable code into acomputing system, in which the code in combination with the computingsystem is capable of performing one or more aspects of the presentinvention.

As yet a further aspect of the present invention, a process forintegrating computing infrastructure comprising integrating computerreadable code into a computer system may be provided. The computersystem comprises a computer readable medium, in which the computermedium comprises one or more aspects of the present invention. The codein combination with the computer system is capable of performing one ormore aspects of the present invention.

Although various embodiments are described above, these are onlyexamples. For example, computing environments of other architectures canincorporate and use one or more aspects of the present invention.Additionally, the network of nodes can include additional nodes, and thenodes can be the same or different from those described herein. Also,many types of communications interfaces may be used.

Further, a data processing system suitable for storing and/or executingprogram code is usable that includes at least one processor coupleddirectly or indirectly to memory elements through a system bus. Thememory elements include, for instance, local memory employed duringactual execution of the program code, bulk storage, and cache memorywhich provide temporary storage of at least some program code in orderto reduce the number of times code must be retrieved from bulk storageduring execution.

Input/Output or I/O devices (including, but not limited to, keyboards,displays, pointing devices, DASD, tape, CDs, DVDs, thumb drives andother memory media, etc.) can be coupled to the system either directlyor through intervening I/O controllers. Network adapters may also becoupled to the system to enable the data processing system to becomecoupled to other data processing systems or remote printers or storagedevices through intervening private or public networks. Modems, cablemodems, and Ethernet cards are just a few of the available types ofnetwork adapters.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprise” (andany form of comprise, such as “comprises” and “comprising”), “have” (andany form of have, such as “has” and “having”), “include” (and any formof include, such as “includes” and “including”), and “contain” (and anyform contain, such as “contains” and “containing”) are open-endedlinking verbs. As a result, a method or device that “comprises”, “has”,“includes” or “contains” one or more steps or elements possesses thoseone or more steps or elements, but is not limited to possessing onlythose one or more steps or elements. Likewise, a step of a method or anelement of a device that “comprises”, “has”, “includes” or “contains”one or more features possesses those one or more features, but is notlimited to possessing only those one or more features. Furthermore, adevice or structure that is configured in a certain way is configured inat least that way, but may also be configured in ways that are notlisted.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below, if any, areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present invention has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The embodiment was chosen and described in order to explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention throughvarious embodiments and the various modifications thereto which aredependent on the particular use contemplated.

For completeness, Table 1 below lists and describes the variablesreferenced herein in describing certain embodiments of the presentinvention.

TABLE 1 Variable Definition DP Dew Point H Humidity P Pressure TTemperature F1 1^(st) Fan or Fan F1 F2 2^(nd) fan or Fan F2 FN − 1 (N −1)th Fan (or Fan FN − 1) FN Nth Fan (or Fan FN) P1 1^(st) Pump or PumpP1 P2 2^(nd) Pump or Pump P2 P2N − 1 (2N − 1)th Pump (or Pump P2N − 1)P2N 2Nth Pump (or Pump P2N) V1 Recirculation valve V1 V2 Recirculationvalve V2 VN − 1 Recirculation valve VN − 1 ((N − 1)th Valve) VNRecirculation valve VN (Nth Valve) ΔRPM_F1 Change in RPM for Fan F1ΔRPM_F2 Change in RPM for Fan F2 ΔRPM_FN − 1 Change in RPM for (N −1)^(th) Fan (or Fan FN − 1) ΔRPM_FN Change in RPM for Nth Fan (or FanFN) ΔRPM_P1 Change in RPM for Pump P1 ΔRPM_P2 Change in RPM for Pump P2ΔRPM_P2N − 1 Change in RPM for (2N − 1)^(th) Pump (or Pump P2N − 1)ΔRPM_P2N Change in RPM for 2N^(th) Pump (or Pump P2N) Δ% Open_V1 Changein % Open for the Recirculation valve V1 Δ% Open_V2 Change in % Open forthe Recirculation valve V2 Δ% Open_VN − 1 Change in % Open for theRecirculation valve VN − 1 ((N − 1)th Valve) Δ% Open_VN Change in % Openfor the Recirculation valve VN (N^(th) Valve) Δ%_V1 Change in % Open forthe Recirculation valve V1 Δ%_V2 Change in % Open for the Recirculationvalve V2 Δ%_VN − 1 Change in % Open for the Recirculation valve VN − 1((N − 1)^(th) Valve) Δ%_VN Change in % Open for the Recirculation valveVN (N^(th) Valve) IZ Z^(th) Input condition Z Total number of inputconditions or inputs I1, 1 Value of input condition 1 at row 1 I1, 2Value of input condition 1 at row 2 I1, r1 Value of input condition 1 atrow r1 I1, r2 Value of input condition 1 at row r2 I1, r Value of inputcondition 1 at row r I2, 1 Value of input condition 2 at row 1 I2, 2Value of input condition 2 at row 2 I2, r3 Value of input condition 2 atrow r3 I2, r4 Value of input condition 2 at row r4 I2, r Value of inputcondition 2 at row r IZ, 1 Value of input condition Z at row 1 IZ, 2Value of input condition Z at row 2 IZ, r Value of input condition Z atrow r F1, 1 Value of Fan F1 RPM at row 1 F1, 2 Value of Fan F1 RPM atrow 2 F1, r Value of Fan F1 RPM at row r F2, 1 Value of Fan F2 RPM atrow 1 F2, 2 Value of Fan F2 RPM at row 2 F2, r Value of Fan F2 RPM atrow r FN, 1 Value of Fan FN RPM at row 1 FN, 2 Value of Fan FN RPM atrow 2 FN, r Value of Fan FN RPM at row r Flag Flag = 1 means reduce RPMsand/or open recirculation valve(s) and Flag = 2 means increase RPMsand/or close recirculation valve(s) P1, 1 Value of Pump P1 RPM at row 1P1, 2 Value of Pump P1 RPM at row 2 P1, r Value of Pump P1 RPM at row rP2, 1 Value of Pump P2 RPM at row 1 P2, 2 Value of Pump P2 RPM at row 2P2, r Value of Pump P2 RPM at row r P2N, 1 Value of Pump P2N RPM at row1 P2N, 2 Value of Pump P2N RPM at row 2 P2N, r Value of Pump P2N RPM atrow r V1, 1 Value of Recirculation Valve V1 % Open at row 1 V1, 2 Valueof Recirculation Valve V1 % Open at row 2 V1, r Value of RecirculationValve V1 % Open at row r V2, 1 Value of Recirculation Valve V2 % Open atrow 1 V2, 2 Value of Recirculation Valve V2 % Open at row 2 V2, r Valueof Recirculation Valve V2 % Open at row r VN, 1 Value of RecirculationValve VN % Open at row 1 VN, 2 Value of Recirculation Valve VN % Open atrow 2 VN, r Value of Recirculation Valve VN % Open at row r M1 Margin bywhich the first constraint is met M2 Margin by which the secondconstraint is met Mz Margin by which the Z^(th) constraint is met ZTotal number of constraints T_(CPU) Hottest CPU or Maximum CPUTemperature T_(CPU,spec) Maximum allowed CPU Temperature T_(DIMM)Hottest DIMM or Maximum DIMM Temperature T_(DIMM,Spec) Maximum allowedDIMM Temperature T_(rack air in) Rack Inlet Air TemperatureT_(rack air in,spec) Maximum allowed Rack Inlet Air TemperatureT_(rack DP) Rack Dew Point Temperature T_(rack DP,spec) Allowed Rack DewPoint Temperature Undo_Flag Undo_Flag = 0 means a change was made toreduce the cooling power. Undo_Flag = 1 means last change was reverted.X2 Relationship variable between ΔRPM_F2 and ΔRPM_F1 X3 Relationshipvariable between ΔRPM_F3 and ΔRPM_F1 XN Relationship variable betweenΔRPM_FN and ΔRPM_F1 Y1 Relationship variable between ΔRPM_P1 and ΔRPM_F1Y2 Relationship variable between ΔRPM_P2 and ΔRPM_F1 Y2N Relationshipvariable between ΔRPM_P2N and ΔRPM_F1 W1 Relationship variable betweenΔ% Open_V1 and ΔRPM_F1 W2 Relationship variable between Δ% Open_V2 andΔRPM_F1 WN Relationship variable between Δ% Open_VN and ΔRPM_F1

What is claimed is:
 1. A method of facilitating dissipation of heat froman electronics rack, the method comprising: associating at least onecoolant-cooled structure with the electronics rack for facilitatingdissipation of heat from the electronics rack, the at least onecoolant-cooled structure comprising at least one coolant-carryingpassage; providing a coolant loop coupled in fluid communication withthe at least one coolant-carrying passage of the at least onecoolant-cooled structure; associating at least one heat exchange unitwith the coolant loop to facilitate heat transfer from coolant withinthe coolant loop; providing N controllable components associated with atleast one of the coolant loop or the at least one heat exchange unit,the N controllable components facilitating at least one of circulatingof coolant through the coolant loop or transfer of heat from the coolantvia the at least one heat exchange unit, wherein N≧1; and providing acontroller coupled to the N controllable components, the controllerdynamically adjusting operation of the N controllable components, basedon Z input parameters and one or more specified constraints, to provideda specified cooling to the at least one coolant-cooled structure, whilelimiting energy consumed by the N controllable components, wherein Z≧1.2. The method of claim 1, wherein the controller dynamically adjustingoperation of the N controllable components comprises the controlleridentifying a set of operational control settings, to be applied to theN controllable components, from a prespecified data structure employingthe Z input parameters, the prespecified data structure comprising aplurality of sets of operational control settings for the N controllablecomponents.
 3. The method of claim 1, wherein the controller dynamicallyadjusting operation of the N controllable components comprises thecontroller ascertaining a margin by which each input parameter of the Zinput parameters meets a respective specified constraint of the one ormore specified constraints, and the controller determining whether aminimum margin of the ascertained margins exceeds a specified minimummargin, and responsive to the minimum margin exceeding the specifiedminimum margin, the controller dynamically reducing one or moreoperational control settings for the N controllable components, andwherein, responsive to the minimum margin of the ascertained marginsbeing below the specified minimum margin, the controller dynamicallyincreases one or more operational control settings for the Ncontrollable components.
 4. The method of claim 1, wherein, responsiveto the minimum margin of the ascertained margins exceeding the specifiedminim margin, the controller dynamically incrementally reduces anoperational control setting for one controllable component of the Ncontrollable components, and upon reaching a minimum operational controlsetting for the one controllable component, the controller dynamicallyincrementally reduces another operational control setting for anothercontrollable component of the N controllable components, and wherein,responsive to the minimum margin of the ascertained margins being belowthe specified minimum margin, the controller dynamically incrementallyincreases an operational control setting for one controllable componentof the N controllable components, and upon reaching a maximumoperational control setting for the one controllable component, thecontroller dynamically incrementally increases another operationalcontrol setting for another controllable component of the N controllablecomponents.
 5. The method of claim 1, wherein the at least one heatexchange unit comprises an outdoor-air-cooled heat exchange unit, theoutdoor-air-cooled heat exchange unit cooling coolant passing throughthe coolant loop by dissipating heat from the coolant to outdoor ambientair.
 6. A cooling method comprising: obtaining an electronics rack withan associated cooling apparatus comprising at least one coolant-cooledstructure associated with the electronics rack for facilitatingdissipation of heat from the electronics rack, the at least onecoolant-cooled structure comprising at least one coolant-carryingpassage, wherein the cooling apparatus further comprises: a coolant loopcoupled in fluid communication with the at least one coolant-carryingpassage of the at least one coolant-cooled structure; at least one heatexchange unit coupled to facilitate heat transfer from the coolantwithin the coolant loop; and N controllable components associated withat least one of the coolant loop or the at least one heat exchange unit,the N controllable components facilitating at least one of circulatingof coolant through the coolant loop or transfer of heat from the coolantvia the at least one heat exchange unit, wherein N≧1; and dynamicallyadjusting operation of the N controllable components, based on Z inputparameters and one or more specified constraints, to provide a specifiedcooling to the at least one coolant-cooled structure, while limitingenergy consumed by the N controllable components, wherein Z≧1.